The present invention relates generally to a sound signal analyzing device and method which, on the basis of a sound signal, such as a voice signal or tone signal inputted via a microphone or the like, having undetermined pitch or note, analyzes sections appearing to have musical sounds and steady sections of the musical sounds so as to automatically analyze the notes (note names in a scale) and note lengths. The present invention also relates to a recording medium storing a program for implementing such operations.
Analyzed results by the present invention can be output as electronic musical staff information such as in the form of MIDI information, and therefore the present invention concerns a technique which permits automatic conversion, into a musical staff, of an audible melody input by human voices or the like.
In recent years, computer music performance systems, which use a computer to generate performance information such as MIDI information and reproduce performance sounds on the basis of the generated performance information, have been attracting people""s attention as new musical sound performance devices. For input of various data to create the performance information, these computer music performance systems employ any of the real-time input method, step input method, numerical value input method, staff input method, etc.
In the real-time input method, information representative of player""s actual operation on a keyboard or other performance operator, which is recorded on a tape recorder or the like, is converted into predetermined performance information on a real time basis. In the numerical value input method, performance information, such as pitches, lengths and strengths of sounds, is input in numerical value data directly from a computer keyboard. In the staff input method, simplified musical note symbols are put in a staff or stave visually presented on a display using function keys or mouse of a computer. In the step input method, musical notes are input using a MIDI keyboard or software keyboard and lengths of sounds are input using function keys or mouse of a computer.
Of the above-mentioned input methods, the real-time input method is advantageous in that it facilitates expression of human feelings and permits rapid input of performance information because the player""s actual performance operation can be recorded directly as performance information. However, this method requires a high-level performance ability or experience on the part of players and hence is not suited to unexperienced players.
Thus, performance information generating devices have been proposed which allow even unexperienced players to readily input performance information while maintaining the advantages of the real-time input method. In the proposed performance information generating devices, a human voice or tone of a natural musical instrument (hereinafter collectively called xe2x80x9csoundsxe2x80x9d) is input directly via a microphone, so as to generate performance information on the basis of the input sound. Namely, by just inputting a single human voice or tone of a natural musical instrument, such as guitar, to the performance information generating device, it can generate MIDI signals in a simple manner and control MIDI equipment without using a MIDI keyboard or the like.
These known performance information generating devices are arranged to generate MIDI information, in response to pitch variation of the sound inputted via the microphone, by use of any one of the following approaches. The first approach is to detect a pitch variation in semitones, so as to generate only note information representative of the detected tone pitch. The second approach is to detect a pitch variation in semitones to generate note information of the detected tone pitch and also generate pitch-bend information (tone pitch varying information). The third approach is to generate pitch bend information variable over one octave above and below the input sound signal without detecting a note. Also, the performance information generating devices compare each input sound level with a predetermined reference value so that it generates note-on information when the input sound level has exceeded the reference value and generates note-off information when the input sound level has lowered below the reference value.
However, where pitch variation is detected in semitones as in the above-mentioned first and second approaches, many unintended note information (note-on or note-off information) would be undesirably generated as the input sound fluctuates in pitch slightly. In addition, the third approach where pitch varying information is generated as pitch bend information is not suited for particular purposes, such as staff making, although intended pitch variation can be faithfully by the pitch bend information. Also, where note information is generated in accordance with the input sound level, many unintended note information would be undesirably generated in response to slight fluctuation in the level.
Furthermore, in the real-time input method, it is necessary efficiently analyze each section where a sound appears to be actually present, because a plurality of sounds are input to a microphone in a time-series at optional time intervals. If, in this case, analysis of pitch and the like is constantly performed on the input sounds, the analysis would be undesirably conducted wastefully even during a time when there is no input sound. Thus, the analysis efficiency could be greatly enhanced by extracting, out of the input sound signals, only sections where sounds appear to be actually present (i.e., available sections) and conducting complicated analysis operations, such as a tone pitch analysis, only for the extracted available sections. Conventionally, such an available section is extracted by merely comparing the input sound signal level with a predetermined reference level, which, however, would present the problem that the available section extraction tends to be inaccurate when the input sound level slightly fluctuates, particularly in the vicinity of the reference level.
It is therefore an object of the present invention to provide a sound signal analyzing device and method which, even when an input sound from a microphone or the like fluctuates slightly in pitch or level, can effectively analyze each steady section of the input sound, other than the fluctuating section, corresponding to a note. More particularly, the present invention provides a technique for effectively analyzing steady sections of a series of input sounds to thereby accurately analyzing respective pitches of the individual sounds.
It is another object of the present invention to provide a sound signal analyzing device and method which, even when an input sound from a microphone or the like fluctuates slightly in pitch or level, can readily analyze an available section of the sound where a musical sound appears to be actually present.
It is still another object of the present invention to provide a performance information generating device which, even when an input sound from a microphone or the like fluctuates slightly in pitch or level, can reliably generate accurate note information corresponding to the pitch of the input sound.
According to a first aspect of the present invention, there is provided a sound signal analyzing device which comprises: an input unit that inputs an optional sound signal to the sound signal analyzing device; an arithmetic operating unit that calculates an average of every predetermined number of sample amplitude values of the sound signal inputted via the input unit and outputs the respective averages as a time-series of average level information; a first section detecting unit that, on the basis of the average level information outputted from the arithmetic operating unit, detects a first section of the inputted sound signal where there appears to be a musical sound; and a second section detecting unit that, on the basis of the sample amplitude values within the first section, detects second sections of the inputted sound signal from the first section for subsequent analysis of the sound signal.
By thus calculating an average of every predetermined number of sample amplitude values of the sound signal inputted via the input unit, there can be obtained average sound pressure level information that smoothly changes sensitively in response to fluctuation in level of the inputted sound signal. Further, because degrees of inclination in the average sound pressure level information are calculated to thereby detect second sections of the inputted sound signal for subsequent analysis of the sound signal, the waveform level in possible same-waveform sections within the sound signal can be constantly stable, which would enhance the efficiency of waveform comparison and also permit reliable detection of same-waveform sections.
In a preferred implementation, the second section detecting unit detects, as a stable-level section, each of the signal sections where the degree of inclination in the average sound pressure level information is smaller than a predetermined value and is greater than a predetermined length, and it detects a second section by expanding such a stable-level section. If the degree of inclination, for a given signal section, in the average sound pressure level information is smaller than the predetermined value but the given signal section is not greater than the predetermined length, that signal section can not be the to be a stable-level section and hence is excluded from further analysis.
A sound signal analyzing device according to another aspect of the present invention comprises: an input unit that inputs an optional sound signal to the sound signal analyzing device; a waveform creating unit that detects a maximum value of every predetermined number of sample amplitude values of the sound signal inputted via the input unit and creates an auxiliary waveform by interpolating between the detected maximum values; a first section detecting unit that, on the basis of the auxiliary waveform created by the waveform creating unit, detects a first section of the inputted sound signal where there appears to be a musical sound; and a second section detecting unit that, on the basis of the sample amplitude values within the first section, detects second sections of the inputted sound signal from the first section for subsequent analysis of the sound signal.
By thus detecting a maximum value of every predetermined number of sample amplitude values of the inputted sound signal and detecting a first section on the basis of an auxiliary waveform obtained by interpolating between the maximum values, the first section detection can be made with highly increased speed.
Preferably, the stable section detecting unit detects the second section by: the second section detecting unit detects the second section by: detecting maximum values of the sample amplitude values of the inputted sound signal by performing envelope detection on the sample amplitude values in opposite directions; interpolating between the detected maximum values to obtain a maximum-value interpolation curve; evaluating total inclinations at individual sample points on the basis of the maximum-value interpolation curve; and detecting, as a stable-level section, a section over some of the sample points where the total inclinations are smaller than a predetermined value and then expanding the stable-level section. By thus performing envelope detection on the sample amplitude values in opposite (forward/rearward) directions, peaks in overtones can be prevented from being erroneously detected as pitch peaks in a waveform of progressively rising level.
A sound signal analyzing device according to another aspect of the present invention comprises: an input unit that inputs an optional sound signal to the sound signal analyzing device; a provisional-periodic-reference-point detecting unit that detects a plurality of provisional periodic reference points of the sound signal inputted via the input unit; a pitch data train generating unit that detects pitches of the inputted sound signal at the provisional periodic reference points detected by the provisional-periodic-reference-point detecting unit and generates a pitch data train indicative of the detected pitches; a filtering unit that performs, on the inputted sound signal, a filtering operation where pass band is controlled to vary over time in accordance with frequencies corresponding to the detected pitches in the pitch data train; a periodic-reference-point detecting unit that detects a plurality of periodic reference points of the sound signal outputted from the filtering unit; a same-waveform-section detecting unit that determines degrees of similarity in waveform between every adjacent signal sections of the inputted sound signal corresponding to the periodic reference points detected by the periodic-reference-point detecting unit and links together the sections having a high degree of similarity to thereby detect same-waveform sections of the inputted sound signal; and a steady section determining unit that determines a steady section of the inputted sound signal on the basis of the same-waveform sections detected by the same-waveform-section detecting unit.
In the sound signal analyzing device arranged in the above-mentioned manner, provisional periodic reference points of the inputted sound signal are detected from the inputted sound signal to detect same-waveform sections; however, if the detected provisional periodic reference points are not correct, it would be difficult to accurately detect same-waveform sections of the inputted sound signal. Thus, this invention is arranged to detect a pitch data train on the basis of the detected provisional periodic reference points and performs a filtering operation where pass band is controlled to vary over time using, as a cut-off frequency, frequencies corresponding to the detected pitches in the pitch data train, so that the inputted sound signal is allowed to approach a sine wave to enable more accurate detection of the provisional periodic reference points of the inputted sound signal. As a result, it is possible to detect same-waveform sections and steady section with highly increased accuracy.
A sound signal analyzing device according to still another aspect of the present invention comprises: an input unit that inputs an optional sound signal to the sound signal analyzing device; a filtering unit that performs, on the sound signal inputted via the input unit, a filtering operation using a predetermined frequency range; a determining unit that determines degrees of similarity in waveform between every adjacent signal sections on the basis of successive sample amplitude values of the inputted sound signal having undergone the filtering operation; a same-waveform-section detecting unit that detects, as same-waveform sections, those of the signal sections having waveforms determined by the determining unit as being similar within a range corresponding to a predetermined condition; and a pitch determining unit that determines a pitch of the sound signal within the same-waveform sections detected by the same-waveform-section detecting unit.
This sound signal analyzing device first detects a stable section of the inputted sound signal and then detects provisional periodic reference points and generates a pitch data train followed by detecting periodic reference points, so as to ultimately detect a steady section of the inputted sound signal. Because relatively stable tone pitch and the like are found in the stable section, detection of a steady section can be made with highly increased speed and accuracy.
A sound signal analyzing device according to still another aspect of the present invention comprises: an input unit that inputs an optional sound signal to the sound signal analyzing device; a provisional-periodic-reference-point detecting unit that detects a plurality of provisional periodic reference points of the sound signal inputted via the input unit; a pitch data train generating unit that detects pitches of the inputted sound signal at the provisional periodic reference points detected by the provisional-periodic-reference-point detecting unit and generates a pitch data train indicative of the detected pitches; a filtering unit that performs, on the inputted sound signal, a filtering operation where pass band is controlled to vary over time in accordance with frequencies corresponding to the detected pitches in the pitch data train; a periodic-reference-point detecting unit that detects a plurality of periodic reference points of the sound signal outputted from the filtering unit; a voiced-sound-containing section detecting unit that determines degrees of similarity in waveform between every adjacent signal sections of the inputted sound signal corresponding to the periodic reference points detected by the periodic-reference-point detecting unit and detects a voiced-sound-containing section of the inputted sound signal on the basis of the calculated degree of similarity; and a steady section determining unit that sequentially calculates degrees of similarity in waveform between a high-similarity basic signal section within the voiced-sound-containing section and other signal sections located to opposite sides of the basic signal section and determines a steady section of the inputted sound signal on the basis of the calculated degrees of similarity.
This sound signal analyzing device determines degrees of similarity in waveform between every adjacent signal sections of the inputted sound signal and determines, as a voiced-sound-containing section of the inputted sound signal, such sections having a degree of similarity greater than a predetermined value. Then, degrees of similarity in waveform are sequentially calculated between a high-similarity basic signal section within the voiced-sound-containing section and other signal sections located to opposite sides of the basic signal section so that a steady section of the inputted sound signal is determined on the basis of the calculated degrees of similarity. Because the high-similarity basic signal section is a basis of a vowel sound, variation in vowel can be detected by the degree of similarity determined by use of the high-similarity basic signal section. The thus-determined steady section can be identified as a vowel, namely, a single note.
The sound signal analyzing device according to still another aspect of the present invention comprises: an input unit that inputs an optional sound signal to the sound signal analyzing device; an available section analyzing unit that analyzes an available section of the sound signal, inputted via the input unit, for subsequent analysis of the sound signal; a provisional-periodic-reference-point detecting unit that detects a plurality of provisional periodic reference points of the sound signal forming the stable section; a pitch data train generating unit that detects pitches of the sound signal at the provisional periodic reference points detected by the provisional-periodic-reference-point detecting unit and generates a pitch data train indicative of the detected pitches; a filtering unit that performs, on the inputted sound signal, a filtering operation where pass band is controlled to vary over time in accordance with frequencies corresponding to the detected pitches in the pitch data train; a periodic-reference-point detecting unit that detects a plurality of periodic reference points of the sound signal outputted from the filtering unit; a voiced-sound-containing section detecting unit that determines degrees of similarity in waveform between every adjacent signal sections of the sound signal corresponding to the periodic reference points detected by the periodic-reference-point detecting unit and detects a voiced-sound-containing section of the sound signal on the basis of the calculated degree of similarity; and a steady section determining unit that sequentially calculates degrees of similarity in waveform between a high-similarity basic signal section within the voiced-sound-containing section and other signal sections located to opposite sides of the basic signal section and determines a steady section of the sound signal on the basis of the calculated degrees of similarity.
This sound signal analyzing device detects a stable section of the inputted sound signal and then detects a voiced-sound-containing section and a steady section within the stable section. Because relatively stable tone pitch and the like are found in the stable section, detection of a steady section can be made with highly increased speed and accuracy.
In a preferred implementation, the provisional-periodic-reference-point detecting unit includes: a first filtering unit that performs, on the sound signal inputted via the input unit, a band-pass filtering operation using predetermined cutoff frequencies as maximum and minimum frequencies; a first periodic-reference-point detecting unit that detects a plurality of provisional periodic reference points of the sound signal outputted from the first filtering unit; a frequency range detecting unit that detects the maximum and minimum frequencies of the sound signal on the basis of the provisional periodic reference points detected by the first periodic-reference-point detecting unit; a second filtering unit that performs, on the sound signal inputted via the input unit, a band-pass filtering operation using as cut-off frequencies the maximum and minimum frequencies detected by the frequency range detecting unit; and a second periodic-reference-point detecting unit that detects a plurality of periodic reference points of the sound signal outputted from the second filtering unit. Namely, such a preferred example of the provisional-periodic-reference-point detecting unit performs two band-pass filtering operations to detect the provisional periodic reference points.
Preferably, the pitch data train generating unit interpolates between pitch data of the sound signal determined at individual ones of the provisional periodic reference points, so as to detect the pitches and generate a data train of the detected pitches. Because the pitch data train is provided through the interpolation operation, the time-varying band-pass filtering can be effected with highly increased accuracy between every adjacent provisional periodic reference points.
Preferably, the provisional-periodic-reference-point detecting unit detects, as the provisional periodic reference points, peak points of the sound signal by focusing on one of plus and minus amplitude sides of a waveform of the sound signal where stronger peaks appear than on another of the plus and minus amplitude sides. With this arrangement, the provisional-periodic-reference-point detecting unit can operate properly even when the inputted sound signal presents more distinct or stronger waveform characteristics on one of the plus and minus sides than on the other.
It is preferable that the periodic-reference-point detecting unit detect, as the periodic reference points, peak points of the sound signal by focusing on one of plus and minus amplitude sides of a waveform of the sound signal where stronger peaks appear than on another of the plus and minus amplitude sides. With this arrangement, the periodic-reference-point detecting unit can operate properly even when the inputted sound signal presents more distinct or stronger waveform characteristics on one of the plus and minus sides than on the other.
It is also preferable that the periodic-reference-point detecting unit divide a waveform of the sound signal into signal sections at predetermined intervals corresponding to the cut-off frequency used in the band-pass filtering operation, by focusing on one of plus and minus amplitude sides of a waveform of the sound signal, having undergone the band-pass filtering operation, where stronger peaks appear than on another of the plus and minus amplitude sides, and wherein the periodic-reference-point detecting unit detects a greatest peak within each of the signal sections as the periodic reference point. By thus dividing the sound signal waveform into signal sections at predetermined intervals corresponding to the cut-off frequency and detecting peak points within each of the divided sections, such peaks occurring at shorter intervals than a predetermined interval can be effectively prevented from being erroneously detected, which permits detection of peak points with highly increased accuracy.
A sound signal analyzing device according to still another aspect of the present invention comprises: an input unit that inputs an optional sound signal to the sound signal analyzing device; a filtering unit that performs, on the sound signal inputted via the input unit, a filtering operation for a predetermined bass band; a peak point detecting unit that detects peak points in the sound signal having undergone the filtering operation by the filtering unit; a same-waveform-section detecting unit that, of signal sections obtained by dividing a waveform of the sound signal at optional pairs of the peak points detected by the peak point detecting unit, selects as many pairs of adjacent signal sections as possible that meet a limit defined by the pass band of the filtering unit, the same-waveform-section detecting unit determining a degree of similarity in waveform between two signal sections of each of the selected pairs and detecting one of the selected pairs having a highest similarity as same-waveform sections; and a steady section determining unit that determines a steady section of the sound signal on the basis of the same-waveform sections detected by the same-waveform-section detecting unit.
This sound signal analyzing device first detects a pair of same-waveform sections on the basis of peak points and then detects subsequent same-waveform sections on the basis of the length of the first detected same-waveform sections, rather than determining, for every pair of the signal sections, whether or not they are similar in waveform in consideration of the pitch length. This arrangement greatly increases the speed of the same-waveform section detection.
A sound signal analyzing device according to still another aspect of the present invention comprises: an input unit that inputs an optional sound signal to the sound signal analyzing device; an input unit that inputs an optional sound signal to the sound signal analyzing device; a peak point detecting unit that detects peak points in the sound signal inputted via the input unit; a first same-waveform-section detecting unit that, of signal sections obtained by dividing a waveform of the sound signal at optional pairs of the peak points detected by the peak point detecting unit, determines degrees of similarity in waveform between every two the signal sections and links together the signal sections having a high similarity so as to detect a first same-waveform section group; a second same-waveform-section detecting unit that, using first and last signal sections in the first same-waveform section group as a basis of comparison, calculates degrees of similarity in waveform between the first same-waveform section group and other signal sections adjoining the first and last signal sections and expands the first same-waveform section group to incorporate one or more of the other signal sections depending on the calculated degrees of similarity, the second same-waveform-section detecting unit detecting the expanded first same-waveform section group as a second same-waveform section group; and a steady section determining unit that determines a steady section of the sound signal on the basis of the second same-waveform section group detected by the second same-waveform-section detecting unit.
In this sound signal analyzing device, a first same-waveform section group detected by the first same-waveform-section detecting unit is expanded by the second same-waveform-section detecting unit. If a criterion used to detect same-waveform sections is very low, detected same-waveform sections tend to be so wide that detection of a steady section becomes difficult; if, on the other hand, the criterion is very high, same-waveform sections tend to be detected only sparsely. Thus, in the present invention, a relatively high criterion to detect same-waveform sections is used in the first same-waveform-section detecting unit, and once a first same-waveform-section detecting unit is detected, it is expanded by the second same-waveform-section detecting unit. This arrangement permits detection of same-waveform sections with highly increased efficiency.
Preferably, if there is any gap signal section that does not belong to either of adjacent second same-waveform sections, degrees of similarity in waveform are evaluated between the last signal section of a preceding one of the adjacent second same-waveform sections and the gap signal section and between the leading signal section of a succeeding one of the adjacent second same-waveform sections and the gap signal section, and the gap signal section is incorporated into one of the adjacent second same-waveform sections to which the gap signal section has a higher degrees of similarity in waveform.
When there is detected a gap signal section that does not belong to either of adjacent second same-waveform sections expanded by the second same-waveform-section detecting unit, this device incorporates it into one of the adjacent same-waveform sections in any one of various ways. For example, the incorporation of the gap signal section may be effected by sequentially comparing degrees of similarity, using, as comparison bases, the last signal section of a preceding one of the adjacent second same-waveform sections and the leading signal section of a succeeding one of the adjacent second same-waveform sections and the gap signal section. Alternatively, a similar operation may be conducted using the incorporated signal section as a last or first section. As another alternative, the gap signal section is not incorporated into either of the adjacent second same-waveform sections if the detected degree of similarity is lower than a predetermined criterion.
A sound signal analyzing device according to still another aspect of the present invention comprises: an input unit that inputs an optional sound signal to the sound signal analyzing device; a provisional-periodic-reference-point detecting unit that detects a plurality of provisional periodic reference points of the sound signal inputted via the input unit; a pitch data train generating unit that detects pitches of the sound signal at the provisional periodic reference points detected by the provisional-periodic-reference-point detecting unit and generates a pitch data train indicative of the detected pitches; a first filtering unit that performs, on the inputted sound signal, a band-pass filtering operation where pass band is controlled to vary over time in accordance with frequencies corresponding to the detected pitches in the pitch data train; a periodic-reference-point detecting unit that detects a plurality of periodic reference points of the sound signal outputted from the first filtering unit; a second filtering unit that performs, on the inputted sound signal, a plurality of filtering operations where pass band is controlled to vary over time in accordance with frequencies corresponding to the detected pitches in the pitch data train and integer multiples of the frequencies, the second filtering unit outputting a sound signal waveform synthesized from various waveforms resultant from the filtering operations; a same-waveform-section detecting unit that determines degrees of similarity in waveform between every adjacent signal sections obtained by dividing the sound signal waveform outputted from the second filtering unit and links together the signal sections having a high similarity so as to detect same-waveform sections of the sound signal; and a steady section determining unit that determines a steady section of the sound signal on the basis of the same-waveform sections detected by the same-waveform-section detecting unit.
This sound signal analyzing device is characterized by performing, on the inputted sound signal, filtering operations where pass band is controlled to vary over time in accordance with frequencies corresponding to the detected pitches in the pitch data train and; integer multiples of the frequencies and detecting same-waveform sections from a sound signal waveform synthesized from various waveforms resultant from the band-pass filtering operations. With this arrangement, the same-waveform-section detection can be made on a sound signal waveform with components of unnecessary frequency ranges removed, and hence the steady section determination can be made with highly increased accuracy.
A sound signal analyzing device according to still another aspect of the present invention comprises: an input unit that inputs an optional sound signal to the sound signal analyzing device; a provisional-periodic-reference-point detecting unit that detects a plurality of provisional periodic reference points of the sound signal inputted via the input unit; a pitch data train generating unit that detects pitches of the sound signal at the provisional periodic reference points detected by the provisional-periodic-reference-point detecting unit and generates a pitch data train indicative of the detected pitches; a first filtering unit that performs, on the inputted sound signal, a filtering operation where pass band is controlled to vary over time in accordance with frequencies corresponding to the detected pitches in the pitch data train; a periodic-reference-point detecting unit that detects a plurality of periodic reference points of the sound signal outputted from the first filtering unit; a second filtering unit that performs, on the inputted sound signal, a filtering operation which is controlled to vary over time in accordance with the detected pitches in the pitch data train in such a manner that a pass band of the filtering operation ranges from fundamental frequencies, corresponding to the detected pitches in the pitch data train, to integer multiples of the fundamental frequencies; a same-waveform-section detecting unit that determines degrees of similarity in waveform between every adjacent signal sections obtained by dividing the sound signal waveform outputted from the second filtering unit and links together the signal sections having a high similarity so as to detect same-waveform sections of the sound signal; and a steady section determining unit that determines a steady section of the sound signal on the basis of the same-waveform sections detected by the same-waveform-section detecting unit.
This sound signal analyzing device is characterized by band-pass filtering operation which is controlled to vary over time in accordance with the detected pitches in the pitch data train in such a manner that a pass band of the filtering operation ranges from fundamental frequencies, corresponding to the detected pitches in the pitch data train, to integer multiples of the fundamental frequencies, and then detecting same-waveform sections from the filtered sound signal waveform. With this arrangement, the same-waveform-section detection can be made on a sound signal waveform with components of unnecessary frequency ranges (outside the pass band) removed, and hence the steady section determination can be made with highly increased accuracy.
A sound signal analyzing device according to still another aspect of the present invention comprises: an input unit that inputs an optional sound signal to the sound signal analyzing device; a periodic-reference-point detecting unit that detects a plurality of periodic reference points of the sound signal inputted via the input unit; a pitch data train generating unit that detects pitches of the sound signal at the periodic reference points detected by the periodic-reference-point detecting unit and generates a pitch data train indicative of the detected pitches; a converting unit that converts differences between every adjacent ones of the detected pitches in the pitch data train into respective relative values based on musical interval representation in cents; a dynamic border calculating unit that calculates a dynamic border on the basis of a dynamic average of the relative values obtained by the converting unit; and a steady section determining unit that detects a stable-pitch steady section by comparing the relative values and the dynamic border calculated by the dynamic border calculating unit.
This sound signal analyzing device is characterized by converting pitches, detected at the individual periodic reference points of the sound signal, into respective relative values based on musical interval representation in cents, calculating a dynamic border on the basis of a dynamic average of the relative values, and then comparing the relative values and the dynamic border so as to determine a steady section of the sound signal. The dynamic average means an average of the relative values at the individual periodic reference points from a predetermined averaging start point to a current point; in other words, the dynamic average is an integral average of relative pitch values up to the current point. This dynamic average is used as a dynamic border, which is a dynamic (i.e., time-varying) boundary values. By creating the dynamic border using the dynamic average of the relative values based on musical interval representation in cents, it is possible to obtain normalized comparison basis data (i.e., dynamic border) for used in detection of a stable-pitch section, and the detecting accuracy can be enhanced. If a musical interval, i.e., relative value, between two adjacent pitches is xe2x80x9c0xe2x80x9d, then the pitches are the same, from which it can be seen that tones of same pitch are sounded in succession. If a musical interval, i.e., relative value, between two adjacent pitches is xe2x80x9c1xe2x80x9d in the case where the relative value xe2x80x9c1xe2x80x9d is assumed to represent a semitone interval, the two pitches differ by a semitone, from which it can be seen that completely different tones of same are in succession. However, in effect, there may occur pitch variation over time even when a same tone is sounded continuously. To deal with such a situation, the dynamic border is used as a determination criterion for detecting a stable-pitch section of such time-varying tones. Thus, once a given signal section suddenly changes from a stable musical interval condition to an instable musical interval condition, it can be determined that the given signal section represents an end of a steady section of the sound signal. On the other hand, when a slight variation in musical interval occurs at a particular area in a stable-musical-interval section, it can be determined that the particular area is not an end of a steady section. As a result, the steady section detection can be made in much the same way as in human ears.
A sound signal analyzing device according to still another aspect of the present invention comprises: an input unit that inputs an optional sound signal to the sound signal analyzing device; a periodic-reference-point detecting unit that detects a plurality of periodic reference points of the sound signal inputted via the input unit; a pitch data train generating unit that detects pitches of the sound signal at the provisional periodic reference points detected by the periodic-reference-point detecting unit and generates a pitch data train indicative of the detected pitches; a converting unit that converts differences between every adjacent ones of the detected pitches in the pitch data train into respective relative values based on musical interval representation in cents; a dynamic border calculating unit that calculates a dynamic border on the basis of a dynamic average of the relative values obtained by the converting unit; a steady section determining unit that detects a stable-pitch steady section by comparing the relative values and the dynamic border calculated by the dynamic border calculating unit; a static border calculating unit that calculates a static border on the basis of a static average of the relative values within the steady section detected by the steady section determining unit; a pitch-determining-section detecting unit that compares the static border and the relative values within the steady section so as to detect a pitch determining section for calculating a representative frequency of the steady section; and a frequency calculating unit that calculates the representative frequency of the steady section on the basis of a pitch data train within the pitch determining section detected by the pitch-determining-section detecting unit.
The static average is a simple arithmetic mean of all the relative values within a steady section and therefore always the same for that steady section. This static average is used as a static border, which is a static boundary value (i.e., comparison basis value) that does not vary over time for the same steady section. If the relative value is smaller than the static border, the pitch-determining-section detecting unit judges that the pitch corresponding to the relative value belongs to a most stable section and determines the most stable section as a pitch determining section. Namely, this sound signal analyzing device is characterized by calculating a representative frequency of the steady section on the basis of pitch data within the most stable section, i.e., pitch determining section according to the static border, rather than performing the calculation for all waveform in the steady section. With this arrangement, a representative frequency of the steady section can be calculated with highly increased accuracy.
The dynamic border calculating unit may calculate the dynamic border using any one of a value obtained by multiplying the dynamic average of the relative values by a predetermined value, a value obtained by adding the predetermined value to the dynamic average of the relative values and a value obtained by adding the predetermined value to the value obtained by multiplying the dynamic average of the relative values by the predetermined value. This permits calculation of a very effective dynamic border.
Once a stable or steady section is detected using one or more of the above-mentioned approaches, a note assigning operation is performed on the thus-detected stable or steady section. Namely, the sound signal analyzing device of the present invention may further comprise a note assigning unit that analyzes a representative frequency of the sound signal and determining notes for the sound signal.
All of the steady sections detected through any of the above-mentioned approach do not always correspond to valid notes. To determine such an xe2x80x9cinvalidxe2x80x9d steady section, the present invention can advantageously employ grids divided at time intervals corresponding to a predetermined note length (e.g., shortest possible note length). Namely, the present invention may further comprise a unit that allots each of the steady sections to one of the grids nearest to a start point thereof, and if a plurality of the steady sections are simultaneously allotted to a particular one of the grids, the unit selects one of the steady sections having a greatest time length as valid. This arrangement determines particular time values of notes to which the detected steady sections should be assigned.
A sound signal analyzing device according to still another aspect of the present invention comprises: an input unit that inputs an optional sound signal to the sound signal analyzing device; an arithmetic operating unit that calculates an average of every predetermined number of sample amplitude values of the sound signal inputted via the input unit and outputs respective the averages as a time-series of average sound pressure level information; a section determining unit that determines each signal section of the sound signal where the average sound pressure level calculated by the arithmetic operating unit is greater than a first predetermined value as an available section where there appears to be a musical sound and determines each other signal section of the sound signal where the average sound pressure level calculated by the arithmetic operating unit is smaller than the first predetermined value as an unavailable section where there appears to be musical sound; an available section adding unit that if any particular one of the unavailable sections located between the available sections is of a time length smaller than a first predetermined length, changes the particular unavailable section into an additional available section and combines the additional available section and the available sections adjoining opposite sides of the additional available section, the available section adding unit determining a combination of the additional available section and adjoining available sections as a new available section; a first unavailable section adding unit that if any particular one of the available sections located between the unavailable sections is of a time length smaller than a second predetermined length after determination by the available section adding unit, changes the particular available section into an additional unavailable section and combines the additional unavailable section and the unavailable sections adjoining opposite sides of the additional unavailable section, the first unavailable section adding unit determining a combination of the additional unavailable section and adjoining unavailable sections as a new unavailable section; and a second unavailable section adding unit that calculates an average of the average sound pressure levels in each of the available sections after determination by the first unavailable section adding unit and that if the calculated average of any particular one of the available sections is smaller than a second predetermined value, changes the particular available section into an additional unavailable section.
By thus calculating an average of every predetermined number of sample amplitude values of the sound signal inputted via the input unit, there can be obtained average sound pressure level information that smoothly changes sensitively in response to fluctuation in level of the inputted sound signal. The thus-obtained average sound levels are classified into available and unavailable sections on the basis of the first predetermined value, and then an ultimate available section is identified on the basis of the first and second predetermined lengths. Thus, even when an inputted sound from a microphone or the like fluctuates slightly in level, the device can effectively analyze the available section where there appears to be a musical sound.
A sound signal analyzing device according to still another aspect of the present invention comprises: an input unit that inputs an optional sound signal to the sound signal analyzing device; an arithmetic operating unit that calculates an average of every predetermined number of sample amplitude values of the sound signal inputted via the input unit and outputs respective the averages as a time-series of average sound pressure level information; a section determining unit that determines each signal section of the sound signal where the average sound pressure level calculated by the arithmetic operating unit is greater than a first predetermined value as an available section, determines each signal section of the sound signal which is located between the available sections and where the average sound pressure level calculated by the arithmetic operating unit is smaller than the first predetermined value as an unavailable section, and also determines each other signal section than the available and unavailable sections as an undetermined section; an available section adding unit that if any particular one of the unavailable sections located between the available sections is of a time length smaller than a first predetermined length, changes the particular unavailable section into an additional available section and combines the additional available section and the available sections adjoining opposite sides of the additional available section, the available section adding unit determining a combination of the additional available section and adjoining available sections as a new available section; a first unavailable section adding unit that if any particular one of the available sections located between the unavailable sections is of a time length smaller than a second predetermined length after determination by the available section adding unit, changes the particular available section into an additional unavailable section and combines the additional unavailable section and the unavailable sections adjoining opposite sides of the additional unavailable section so that the first unavailable section adding unit determines a combination of the additional unavailable section and adjoining unavailable sections as a new unavailable section, and that if any particular one of the available sections adjoining the undetermined section is of a time length smaller than the second predetermined length after determination by the available section adding unit, combines the particular available section and the unavailable and undetermined sections adjoining the particular available section so that the first unavailable section adding unit determines a combination of the particular available section and the unavailable and undetermined sections adjoining the particular available section as a new undetermined section; and a second unavailable section adding unit that calculates an average of the average sound pressure levels in each of the available and undetermined sections after determination by the first unavailable section adding unit and that if the calculated average of any particular one of the available and undetermined sections is smaller than a second predetermined value, changes the particular available or undetermined section into an additional unavailable section, but, if the calculated average of any particular one of the available and undetermined sections is greater than the second predetermined value, changes the undetermined section into an additional available section.
This sound signal analyzing device is characterized by classifying the rising and falling regions of obtained average sound levels as undetermined sections when classifying the sound levels into available and unavailable sections on the basis of the first predetermined value. This arrangement can thus accurately determine whether the rising and falling regions are available sections or not.
A sound signal analyzing device according to still another aspect of the present invention comprises: an input unit that inputs an optional sound signal to the sound signal analyzing device; a periodic-reference-point detecting unit that detects a plurality of periodic reference points of the sound signal inputted via the input unit; a same-waveform-section detecting unit that determines degrees of similarity in waveform between every adjacent signal sections of the sound signal corresponding to the periodic reference points detected by the periodic-reference-point detecting unit and links together the signal sections having a high similarity so as to detect same-waveform sections of the sound signal; and a steady section determining unit that determines a steady section of the sound signal on the basis of the same-waveform sections detected by the same-waveform-section detecting unit.
A sound signal analyzing device according to still another aspect of the present invention comprises: an input unit that inputs an optional sound signal to the sound signal analyzing device; a first periodic-reference-point detecting unit that detects a plurality of provisional periodic reference points of the sound signal inputted via the input unit; a frequency range detecting unit that detects maximum and minimum frequencies of the sound signal on the basis of the provisional periodic reference points detected by the first periodic-reference-point detecting unit; a filtering unit that performs, on the sound signal, a band-pass filtering operation using as cut-off frequencies the maximum and minimum frequencies detected by the frequency range detecting unit; a second periodic-reference-point detecting unit that detects a plurality of periodic reference points of the sound signal outputted from the filtering unit; a same-waveform-section detecting unit that determines degrees of similarity in waveform between every adjacent one of signal sections of the sound signal corresponding to the periodic reference points detected by the second periodic-reference-point detecting unit and links together the signal sections having a high similarity so as to detect same-waveform sections of the sound signal; and a steady section determining unit that determines a steady section of the sound signal on the basis of the same-waveform sections detected by the same-waveform-section detecting unit.
This sound signal analyzing device is characterized by detecting a plurality of provisional periodic reference points of the sound signal, detecting maximum and minimum frequencies of the sound signal on the basis of the provisional periodic reference points, and then performing a band-pass filtering operation using as cut-off frequencies the maximum and minimum frequencies. The band-pass filtering operation can effectively remove unnecessary low-frequency components and harmonics that would lead to errors in detecting same-waveform sections, so that the steady section analysis can be made with highly increased accuracy.
A sound signal analyzing device according to still another aspect of the present invention comprises: an input unit that inputs an optional sound signal to the sound signal analyzing device; an available section analyzing unit that analyzes an available section of the sound signal, inputted via the input unit, where there appears to be a musical sound; a periodic-reference-point detecting unit that detects a plurality of periodic reference points on plus and minus amplitude sides of the sound signal forming the available section; a same-waveform-section detecting unit that for each of the plus and minus amplitude sides of the sound signal, determines degrees of similarity in waveform between every adjacent signal sections of the sound signal corresponding to the periodic reference points detected by the periodic-reference-point detecting unit and links together the signal sections having a high similarity so as to detect same-waveform sections of the sound signal; a tone-color-section determining unit that determines, as same-tone-color sections, signal sections obtained by superposing the plus and minus amplitude sides of the same-waveform sections detected by the same-waveform-section detecting unit; and a steady section determining unit that determines a steady section of the sound signal on the basis of the same-tone-color sections determined by the tone-color-section determining unit.
By thus detecting a plurality of periodic reference points on plus and minus amplitude sides of the sound signal, detecting same-waveform sections on the basis of the detected periodic reference points and then superposing the plus and minus amplitude sides of the same-waveform sections to determine same-waveform sections, detection errors can be minimized even when the sound signal fluctuates slightly in pitch and level on the plus and minus amplitude sides. On the basis of sudden changes in pitch and sound pressure in the thus-determined same-tone-color sections, each steady section is analyzed which corresponds to a single note. Thus, even when an inputted sound from a microphone or the like fluctuates slightly in pitch or level, it is possible to effectively analyze each steady section of a musical sound other than the fluctuating section, i.e., section corresponding to a single note.
A sound signal analyzing device according to still another aspect of the present invention comprises: an input unit that inputs an optional sound signal to the sound signal analyzing device; an available section analyzing unit that analyzes an available section of the sound signal, inputted via the input unit, where there appears to be a musical sound; a first periodic-reference-point detecting unit that detects a plurality of provisional periodic reference points of the sound signal forming the available section; a frequency range detecting unit that detects maximum and minimum frequencies of the sound signal on the basis of the provisional periodic reference points detected by the first periodic-reference-point detecting unit; a filtering unit that performs, on the sound signal, a band-pass filtering operation using as cut-off frequencies the maximum and minimum frequencies detected by the frequency range detecting unit; a second periodic-reference-point detecting unit that detects a plurality of periodic reference points of the sound signal outputted from the filtering unit; a same-waveform-section detecting unit that, for each of plus and minus amplitude sides of the sound signal, determines degrees of similarity in waveform between every adjacent one of signal sections of the sound signal corresponding to the periodic reference points detected by the second periodic-reference-point detecting unit and links together the signal sections having a high similarity so as to detect same-waveform sections of the sound signal; and a steady section determining unit that determines a steady section of the sound signal on the basis of the same-waveform sections detected by the same-waveform-section detecting unit.
This sound signal analyzing device is characterized by detecting a plurality of provisional periodic reference points of the sound signal, detecting maximum and minimum frequencies of the sound signal on the basis of the provisional periodic reference points, and then performing a band-pass filtering operation using as cut-off frequencies the maximum and minimum frequencies. The band-pass filtering operation can effectively remove unnecessary low-frequency components and harmonics that would lead to errors in detecting same-waveform sections, so that the steady section analysis can be made with highly increased accuracy.
According to yet another aspect of the present invention, there is provided a performance information generating device which comprises: an input unit that inputs an optional sound signal to the performance information generating device; a steady section analyzing unit that analyzes a steady section, of the sound signal inputted via the input unit, corresponding to a single note; a frequency range determining unit that determines a representative frequency of each of the steady sections analyzed by the steady section analyzing unit; a converting unit that converts differences in the representative frequency between every adjacent ones of the steady sections into relative values based on musical interval representation in cents; a musical interval data creating unit that creates musical interval data indicative of a musical interval between the adjacent steady sections on the basis of the corresponding relative value; and a note assigning unit that assigns respective notes of a predetermined scale to the steady sections on the basis of the corresponding musical interval data.
This performance information generating device is characterized by determining a representative frequency of each of the analyzed steady sections, creating musical interval data indicative of a musical interval between adjacent steady sections on the basis of a difference in the representative frequency between the adjacent steady sections based on musical interval representation in cents, and then assigning respective notes of a predetermined scale to the steady sections on the basis of the musical interval data. The representative frequency of each of the steady sections is an average of a plurality of waveforms forming that steady section, and the musical interval data is created on the basis of a relative value representing a difference in the representative frequency between two adjacent steady sections. Thus, even when an inputted sound from a microphone or the like fluctuates slightly in pitch, resultant error components can be absorbed in ultimately assigned notes of a scale.
A performance information generating device according to another aspect of the present invention comprises: an input unit that inputs an optional sound signal to the performance information generating device; a steady section analyzing unit that analyzes a steady section, of the sound signal inputted via the input unit, corresponding to a single note; a frequency range determining unit that determines a representative frequency of each of the steady sections analyzed by the steady section analyzing unit; a phrase detecting unit that combines a plurality of the steady sections analyzed by the steady section analyzing unit to detect a single phrase; a converting unit that converts a difference in the representative frequency between each of the steady sections within the phrase detected by the phrase detecting unit and every other steady section preceding the steady sections within the phrase, into a relative value based on musical interval representation in cents; a weighing unit that, for each of the steady sections within the phrase detected by the phrase detecting unit, calculates a weight based on a time distance relative to every other steady section preceding the steady section; a musical interval data calculating unit that, for each of the steady sections, calculates musical interval data indicative of a musical interval from another steady section on the basis of the corresponding relative value obtained by the converting unit and the corresponding weight calculated by the weighing unit; and a note assigning unit that assigns respective notes of a predetermined scale to the steady sections on the basis of the corresponding musical interval data.
This performance information generating device is characterized by, for a phrase formed by a plurality of steady sections, determining a representative frequency and relative value based on musical interval representation in cents, weighting each of the steady section on the basis of a time distance relative to every other steady section preceding that steady section to calculate musical interval data, and then assigning respective notes of a predetermined scale to the steady sections on the basis of the corresponding musical interval data. Thus, even when an inputted sound from a microphone or the like fluctuates slightly in pitch, it is possible to carry out proper note assignment corresponding to respective tones of steady sections forming a phrase.
A performance information generating device according to still another aspect of the present invention comprises: an input unit that inputs an optional sound signal to the performance information generating device; a steady section analyzing unit that analyzes a steady section, of the sound signal inputted via the input unit, corresponding to a single note; a frequency range determining unit that determines a representative frequency of each of the steady sections analyzed by the steady section analyzing unit; a phrase detecting unit that combines a plurality of the steady sections analyzed by the steady section analyzing unit to detect a single phrase; a converting unit that converts a difference in the representative frequency between a leading one of the steady sections within the phrase detected by the phrase detecting unit and every other steady section succeeding the leading steady section, into a relative value based on musical interval representation in cents; a musical interval data calculating unit that, for each of the steady sections, calculates musical interval data indicative of a musical interval from the leading steady section on the basis of the corresponding relative value obtained by the converting unit; and a note assigning unit that assigns respective notes of a predetermined scale to the steady sections on the basis of the corresponding musical interval data.
For a phrase formed by a plurality of steady sections, this device determines a representative frequency of each of the steady section, calculates musical interval data indicative of a musical interval between the leading steady section and every other steady section, and assigns respective notes of a predetermined scale to the steady sections on the basis of the musical interval data. Thus, even when an inputted sound from a microphone or the like fluctuates slightly in pitch, it is possible to carry out proper note assignment corresponding to the leading tone of the phrase.
The note assigning unit may analyze a representative frequency of the sound signal for each of the steady sections analyzed by the steady section analyzing unit and then assign respective notes of a predetermined scale to the steady sections on the basis of analyzed results. At that time, the note assigning unit may first assigns a predetermined note of the predetermined scale to a leading one of the steady sections and then sequentially assign a note of the predetermined scale to every other steady section.
Further, in a preferred implementation, the note assigning unit analyzes a representative frequency of the sound signal for each of the steady sections analyzed by the steady section analyzing unit and then assigns respective notes of a predetermined scale to the steady sections on the basis of analyzed results. At that time, the note assigning unit may first analyzes a leading one of the steady sections to detect an average frequency of the leading steady section, then assign a predetermined note, based on the detected average frequency, of the predetermined scale to the leading steady section and then sequentially assign a note of the predetermined scale to every other steady section.
In another preferred implementation, the note assigning unit analyzes a representative frequency of the sound signal for each of the steady sections analyzed by the steady section analyzing unit and assigns respective notes of a predetermined scale to the steady sections on the basis of analyzed results. At that time, the note assigning unit may first provisionally assign respective notes of a plurality of scales to the steady sections while deviating note positions from each other so as to calculate cumulative total note assignment differences at the individual note positions of the scales and then determines an optimum scale on the basis of the calculated cumulative total note assignment differences so as to sequentially assign respective notes of the determined optimum scale to the steady sections.
In still another preferred implementation, the note assigning unit analyzes a representative frequency of the sound signal for each of the steady sections analyzed by the steady section analyzing unit and selects a predetermined scale on the basis of analyzed results so as to assigns respective notes of the predetermined scale to the steady sections. The note assigning unit may also be arranged to assign a note, other than the notes of the predetermined scale, depending on a predetermined note difference allowance.